Location, location, location

Thanks to Pylyshyn's work in the imagery debate, theorists are now apt to be at least a bit more cautious when they launch into descriptions of depictive representation. Even if the full force of his critique is not everywhere acknowledged, more care and vigilance is exercized when broaching talk of scanning mental images, focusing on such entities, or rotating them. The very places found in mental images--places that once rested so serenely under the gaze of the mind's eye--now tend to arouse feelings of disquietude, if not alarm. In this paper I will argue that there is an admirable theoretical continuity between Pylyshyn's critique of pictorial representation in mental imagery and his critique of "location based" models in visual perception and visual selective attention. If it is not appropriate these days for decent minds to look at inner pictures, is it any more appropriate for them to move the spotlight of attention across the master map? If our talk of places in mental images is bankrupt, then what are we to make of our talk of feature maps in the brain? Might not much of that real estate get foreclosed as well? Pylyshyn suggests the answer is yes, and proposes we reallocate now, into object-based alternatives. I shall argue the situation is not quite so dire. Following the analytical lead of P. J. O'Rourke, I shall propose a four way classification. There are indeed Bad Locations, but there are also Good Locations. Similarly, there are Good Objects, but there are also Bad Objects. I hope to clarify some of the distinctions between these. 1. Imagery, Round One To understand Pylyshyn on perception, it is useful, and perhaps essential, first to understand his contributions on what might seem to be a distinct topic: mental imagery. The 1980's imagery debate was a portentous one for mental pictures, and Pylyshyn played a decisive role in it. Many of his recent (2001, 2003) arguments about the architecture of visual perception, and against "location-based" models, show a striking and admirable continuity with those earlier arguments about the forms of representation implicated in mental imagery. As he puts it near the beginning of his recent book: we must dispense with the "picture in the head" ... we must also revise our ideas concerning the nature of the mechanisms involved in vision and concerning the nature of the internal informational states corresponding to percepts or images. (Pylyshyn 2003, 3) In the imagery debate we had bad inferences from experimental data to claims for a distinct, pictorial form of representation. Some of those same patterns of inference are found as well in the "objects v. locations" debate in visual perception. What is the bad pattern of inference? The fundamental issue is: Do any available experimental results entitle us to believe that subjects in imagery tasks use a form of representation that is distinct in kind from the forms used in linguistic tasks? Do they provide any reason at all to think this? Pylyshyn says, forthrightly and firmly, "no". The question is whether results establish use of a distinct form of representation: of a "pictorial" or "depictive" form, as opposed to a "propositional" variety. To do this results must be traceable to a feature of the cognitive architecture, not simply to implicit knowledge, task demands, strategies, or some other labile cause. What would it be to manifest a depictive form? Let us try to be clear on what we take to be the central issue: does visual mental imagery rely (in part) on a distinct type of representation, namely, one that depicts rather than describes? By "depict" we mean that each portion of the the representation is a representation of a portion of the object such that the distances among portions of the representation correspond to the distances among the corresponding portions of the object (as seen from a specific point of view; see Kosslyn 1994...) (Kosslyn, Thompson & Ganis 2002, 198) A depictive representation is a type of picture, which specifies the locations and values of configurations of points in a space. ... In a depictive representation, each part of an object is represented by a pattern of points, and the spatial relation among these patterns in the functional space correspond to the spatial relations among the parts themselves. Depictive representations convey meaning via their resemblance to an object, with parts of the representation corresponding to parts of the object... (Kosslyn 1994, 5) Pylyshyn's position: what I shall argue is not true is that the information in the visual store is pictorial in any sense; i.e., the stored information does not act as though it is a stable and reconstructed extension of the retina. (Pylyshyn 2003, 15) AUSTEN CLARK LOCATION, LOCATION, LOCATION 2 In the opinion of this spectator, the first round of the imagery debate ended roughly as follows. Two widespread, deep, and stubborn sets of reasons for holding to the pictorial form were by Pylyshyn isolated, illuminated, targeted, terminated, dissected, sliced, stained, and mounted. What was left was taken out back and buried. Unfortunately, those scraps seem to reanimate; they don't stay buried for long. The two, seemingly immortal, irrepressible reasons for mental pictures were (and are), first, that introspection reveals the pictorial form directly. The experience of having a mental image is like the experience of seeing something spread out in front of you. How can you deny that you seem to be looking at a picture? A good lawyer could make any witness who denies such a thing seem (at the very least) disingenuous; more likely a scoundrel and a liar, deserving to be convicted. Second, the intentionalist fallacy. When we talk about "the image" it can become almost impossible to tell whether we are talking about the thing imagined or the thing that does the imagining. Mental pictures suffer from the same queasy ambiguity. But in straightforward contexts, at least, it is straightforward: places in the things one represents need not be represented by places in one's representings. If we carefully avoid these two mistakes, what is left of the argument for the claim that mental imagery must employ a distinct pictorial form? Not much. Pylyshyn also provided many arguments in detail about the inadequacies of "depictive" models. The most potent: that the content of the image depends on the subject's beliefs about the objects in the domain in question. 2. Imagery, Round Two Round two of the imagery debate opened with the publication in 1994 of Stephen Kosslyn's Image and Brain, optimistically subtitled The Resolution of the Imagery Debate. (The analogy that springs to mind is a philosopher proposing a final resting place for zombies.) Accounts of depictive representation are amended, and the arguments acquire a neuroscience garnish. The key amendment is that the spatial properties and relations of the image are now construed as properties and relations in a "functional space". The basic idea: talk of spatial properties and relations ascribed to the image should not be taken literally. Instead all those attributions are a kind of "as if" talk, where what we're really talking about are the values returned by the procedures that read, write, and manipulate information in the image. Those procedures function in a way that is analogous to operations applied to a literal two dimensional display. If the image is an array in a computer, we have procedures that access and manipulate distances between points. Those distances (the values returned by these procedures) would be true of a literal two dimensional surface. But this doesn't require that values of adjacent cells in the array be physically next to one another. Basically this is a move to Roger Shepard's idea of second order isomorphism: the image models spatial relations, but it need not itself employ spatial relations to do so. Second, and more importantly for my purposes, neuroscience is claimed to provide evidence for some key features of depictions. First, that visual mental imagery uses some of the same brain mechanisms as does visual perception (in particular V1), and second, that neuroscience shows that those mechanisms use depictive representation. Kosslyn says: Without question, topographically organized cortical areas support depictive representations that are used in visual perception. These areas are not simply physically topographically organized, they function to depict information. For example, scotomas--blind spots--arise following damage to topographically organized visual cortex; damage to nearby regions of cortex results in blind spots that are nearby in the visual field. Moreover, transcranial magnetic stimulation of nearby occipital cortical sites produces phosphenes or scotomas localized at nearby locations in the visual field. These facts testify that topographically organized areas do play a key role in vision, and that they functionally depict information. (Kosslyn, Thompson & Ganis 2002, 200) the actual physical wiring is designed to "read" the depictive aspects of the representation in early visual cortex. In so doing, the interpretive function is not arbitrary; it is tailor made for the representation, which is depictive. (Kosslyn, Thompson & Ganis 2002, 199) What defines round two as qualitatively distinct from round one is this appeal to neuroscience: the reference to topographically organized "feature maps", conjoined to the claim that some of the same mechanisms could support visual imagery. Now the appeal to neuroscience adds yet another kind of image to the already confusing mix (fMRI images of the brain), and yet another kind of map ("feature maps"). If we can avoid being distracted by these pictures, however, the critical premise is easy to spot: that "topographically organized cortical areas support depictive representations". What are we to make of this premise? Pylyshyn gives a characteristically forthright response: Even if we found real colored stereo pictures displayed on the visual cortex, the problems raised thus far in this and the previous chapter would remain and would continue to stand as evidence that these AUSTEN CLARK LOCATION, LOCATION, LOCATION 3 cortical pictures were not serving the function attributed to them. (Pylyshyn 2003, 388) The scraps have reanimated and reorganized; the debate is up and running, once again. And with that I can state the point of this paper. Theoretical objections to "depictive" representation, if they are cogent, would apply not just to imagery, but to everything, including visual perception. So, in particular, they would seem to rule out certain accounts of "location based" effects in selective attention. If places in a mental picture are problematic, what are we to make (for example) of the notion of a "spotlight of attention" moving across the "master map", traversing intermediary locations as it moves, in its own inscrutable fashion, from A to B? For this to make sense we need places which the spotlight traverses, or across which the "window of attention" moves. Such places have alarming similarities to those found in mental images. How, if at all, can we make sense of the locations posited in locationbased models? Perhaps the very notion of a "feature map" is at risk. Does any and every account of feature maps endorse some sort of "inner picture" model? In what sense, if any, are "feature maps" maps? My goal in what remains is to sort some theoretical commitments on these topics into two bins: good and bad. The task is necessary and unpleasant. Theorists must sort out which aspects of an analogical model apply to the real system, and which do not. Here our analogical model for a visual state is a picture or a road map. When we talk of feature maps as "maps", which of the properties of maps must be taken literally? Which are meant only as metaphors? The task can be unpleasant, but I hope here to render it less so by following the analytical lead of P. J. O'Rourke in his masterpiece of economic analysis, Eat the Rich. O'Rourke (1998, 1) says: "I had one fundamental question about economics: Why do some places prosper and thrive while others just suck?" Why indeed? The question applies to visual places too. O'Rourke follows this question with four chapters, entitled Good Capitalism, Bad Capitalism, Good Socialism, Bad Socialism. Here I shall try to distinguish Good Objects from Bad Objects, and Good Locations from Bad Locations. Because Pylyshyn's critique focuses on the badness of Bad Locations, I shall start there.